Method and device for decoding image in image coding system

ABSTRACT

An image decoding method performed by a decoding device according to the present invention comprises the steps of: acquiring information on inter prediction of a target block through a bit stream; deriving motion information of the target block on the basis of the information on inter prediction; deriving a prediction sample by performing inter prediction for the target block on the basis of the motion information; generating a reconstruction block on the basis of the prediction sample; deriving modified motion information of the target block on the basis of the reconstruction block; and updating the motion information of the target block on the basis of the modified motion information. According to the present invention, modified motion information of the target block can be calculated, and the motion information of the target block can be updated on the basis of the modified motion information, so that the overall coding efficiency can be improved.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is the National Stage filing under 35 U.S.C. 371 ofInternational Application No. PCT/KR2016/014167, filed on Dec. 5, 2016,the contents of which are hereby incorporated by reference herein intheir entirety.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to a technique for image coding and, moreparticularly, to a method and apparatus for decoding an image in animage coding system.

Related Art

Demand for high-resolution, high-quality images such as HD (HighDefinition) images and UHD (Ultra High Definition) images have beenincreasing in various fields. As the image data has high resolution andhigh quality, the amount of information or bits to be transmittedincreases relative to the legacy image data. Therefore, when image datais transmitted using a medium such as a conventional wired/wirelessbroadband line or image data is stored using an existing storage medium,the transmission cost and the storage cost thereof are increased.

Accordingly, there is a need for a highly efficient image compressiontechnique for effectively transmitting, storing, and reproducinginformation of high resolution and high quality images.

SUMMARY OF THE INVENTION

The present invention provides a method and apparatus for improvingimage coding efficiency.

The present invention also provides a method and apparatus for aninter-prediction that updates motion information of a target block.

The present invention also provides a method and apparatus forcalculating modified motion information of a target block after decodingprocedure of the target block, and updating based on the modified motioninformation.

The present invention also provides a method and apparatus for usingupdated motion information of a target block for motion information of anext block neighboring the target block.

In an aspect, an image decoding method performed by a decoding apparatusis provided. The image decoding method includes obtaining an informationon inter prediction of a target block through a bitstream, derivingmotion information of the target block based on the information on theinter prediction, deriving a prediction sample by performing the interprediction for the target block based on the motion information,generating a reconstructed block based on the prediction sample,deriving modified motion information for the target block based on thereconstructed block, and updating the motion information of the targetblock based on the modified motion information.

In another aspect, a decoding apparatus for performing an image decodingis provided. The decoding apparatus includes an entropy decoding unitconfigured to obtain information on inter prediction of a target blockthrough a bitstream, a prediction unit configured to deriving motioninformation of the target block based on the information on the interprediction, derive a prediction sample by performing the interprediction for the target block based on the motion information,generate a reconstructed block of the target block based on theprediction sample and derive modified motion information for the targetblock based on the reconstructed block, and a memory configured toupdate the motion information of the target block based on the modifiedmotion information.

In still another aspect, a video encoding method performed by anencoding apparatus is provided. The method includes generating motioninformation for a target block, deriving a prediction sample byperforming inter prediction for the target block based on the motioninformation, generating a reconstructed block based on the predictionsample, generating modified motion information for the target blockbased on the reconstructed block, and updating the motion information ofthe target block based on the modified motion information.

In still another aspect, a video encoding apparatus is provided. Theencoding apparatus includes a prediction unit configured to generatemotion information for a target block, derive a prediction sample byperforming inter prediction for the target block based on the motioninformation, generate a reconstructed block based on the predictionsample, and generate modified motion information for the target blockbased on the reconstructed block, and a memory configured to update themotion information of the target block based on the modified motioninformation.

According to the present invention, after decoding procedure of a targetblock, modified motion information of the target block is calculated,and can be updated to more accurate motion information, and throughthis, overall coding efficiency can be improved.

According to the present invention, motion information of a next blockneighboring the target block can be derived based on the updated motioninformation of the target block, and propagation of distortion can bedecreased, and through this, overall coding efficiency can be improved.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram illustrating a configuration of a videoencoding device to which the present disclosure is applicable.

FIG. 2 is a schematic diagram illustrating a configuration of a videodecoding device to which the present disclosure is applicable.

FIG. 3 illustrates an example of a case of performing an interprediction based on uni-directional motion information and a case ofperforming an inter prediction based on bi-directional motioninformation which is applied to a target block.

FIG. 4 illustrates an example of an encoding procedure including amethod of updating the motion information of a target block.

FIG. 5 illustrates an example of decoding procedure including a methodof updating the motion information of a target block.

FIG. 6 illustrates an example of a method of updating the motioninformation of the target block through the block matching method by thedecoding apparatus.

FIG. 7 illustrates reference pictures which can be used in the modifiedmotion information.

FIG. 8 illustrates an example of a merge candidate list of a next blockneighboring the target blocks in the case that the motion information ofthe target blocks is updated to the modified motion information of thetarget blocks.

FIG. 9 illustrates an example of a merge candidate list of a next blockneighboring the target blocks in the case that the motion information ofthe target blocks is updated to the modified motion information of thetarget blocks.

FIG. 10 illustrates an example of a motion vector predictor candidatelist of a next block neighboring the target block in the case that themotion information of the target block is updated based on the modifiedmotion information of the target block.

FIG. 11 illustrates an example of a motion vector predictor candidatelist of a next block neighboring the target block in the case that themotion information of the target block is updated based on the modifiedmotion information of the target block.

FIG. 12 schematically illustrates a video encoding method by an encodingapparatus according to the present invention.

FIG. 13 schematically illustrates a video decoding method by a decodingapparatus according to the present invention.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

The present disclosure may be modified in various forms, and specificembodiments thereof will be described and illustrated in the drawings.However, the embodiments are not intended for limiting the disclosure.The terms used in the following description are used to merely describespecific embodiments, but are not intended to limit the disclosure. Anexpression of a singular number includes an expression of the pluralnumber, so long as it is clearly read differently. The terms such as“include” and “have” are intended to indicate that features, numbers,steps, operations, elements, components, or combinations thereof used inthe following description exist and it should be thus understood thatthe possibility of existence or addition of one or more differentfeatures, numbers, steps, operations, elements, components, orcombinations thereof is not excluded.

On the other hand, elements in the drawings described in the disclosureare independently drawn for the purpose of convenience for explanationof different specific functions, and do not mean that the elements areembodied by independent hardware or independent software. For example,two or more elements of the elements may be combined to form a singleelement, or one element may be divided into plural elements. Theembodiments in which the elements are combined and/or divided belong tothe disclosure without departing from the concept of the disclosure.

Hereinafter, embodiments of the present disclosure will be described indetail with reference to the accompanying drawings. In addition, likereference numerals are used to indicate like elements throughout thedrawings, and the same descriptions on the like elements will beomitted.

In the present specification, generally a picture means a unitrepresenting an image at a specific time, a slice is a unit constitutinga part of the picture. One picture may be composed of plural slices, andthe terms of a picture and a slice may be mixed with each other asoccasion demands.

A pixel or a pel may mean a minimum unit constituting one picture (orimage). Further, a “sample” may be used as a term corresponding to apixel. The sample may generally represent a pixel or a value of a pixel,may represent only a pixel (a pixel value) of a luma component, and mayrepresent only a pixel (a pixel value) of a chroma component.

A unit indicates a basic unit of image procedureing. The unit mayinclude at least one of a specific area and information related to thearea. Optionally, the unit may be mixed with terms such as a block, anarea, or the like. In a typical case, an M×N block may represent a setof samples or transform coefficients arranged in M columns and N rows.

FIG. 1 briefly illustrates a structure of a video encoding device towhich the present disclosure is applicable.

Referring to FIG. 1, a video encoding device 100 includes a picturepartitioner 105, a predictor 110, a subtractor 115, a transformer 120, aquantizer 125, a re-arranger 130, an entropy encoder 135, a dequantizer140, an inverse transformer 145, an adder 150, a filter 255, and amemory 160.

The picture partitioner 105 may split an input picture into at least oneprocedureing unit. Here, a procedureing unit may be a coding unit (CU),a prediction unit (PU) or a transform unit (TU). The coding unit is aunit block of coding, and a largest coding unit (LCU) may be split intocoding units of deeper depth according to a quad-tree structure. In thiscase, the largest coding unit may be used as a final coding unit or acoding unit may be recursively split into coding units of deeper depthas necessary and a coding unit having an optimal size may be used as afinal coding unit based on coding efficiency according to videocharacteristics. When a smallest coding unit (SCU) is set, a coding unitcannot be split into a coding unit smaller than the smallest codingunit. Here, the final coding unit refers to a coding unit partitioned orsplit into a predictor or a transformer. A prediction unit is a blockpartitioned from a coding unit block and may be a unit block of sampleprediction. Here, the prediction unit may be divided into sub blocks. Atransform block can be split from a coding unit block according to thequad-tree structure and may be a unit block that derives a transformcoefficient and/or a unit block that derives a residual signal from atransform coefficient.

Hereinafter, the coding unit may be called a coding block (CB), theprediction unit may be called a prediction block (PB), and the transformunit may be called a transform block (TB).

The prediction block or the prediction unit may mean a specific areahaving a block shape in a picture, and may include an array of aprediction sample. Further, the transform block or the transform unitmay mean a specific area having a block shape in a picture, and mayinclude a transform coefficient or an array of residual samples.

The predictor 110 may perform prediction on a procedureing target block(hereinafter, a current block), and may generate a prediction blockincluding prediction samples for the current block. A unit of predictionperformed in the predictor 110 may be a coding block, or may be atransform block, or may be a prediction block.

The predictor 110 may determine whether intra-prediction is applied orinter-prediction is applied to the current block. For example, thepredictor 110 may determine whether the intra-prediction or theinter-prediction is applied in unit of CU.

In case of the intra-prediction, the predictor 110 may derive aprediction sample for the current block based on a reference sampleoutside the current block in a picture to which the current blockbelongs (hereinafter, a current picture). In this case, the predictor110 may derive the prediction sample based on an average orinterpolation of neighboring reference samples of the current block(case (i)), or may derive the prediction sample based on a referencesample existing in a specific (prediction) direction as to a predictionsample among the neighboring reference samples of the current block(case (ii)). The case (i) may be called a non-directional mode or anon-angular mode, and the case (ii) may be called a directional mode oran angular mode. In the intra-prediction, prediction modes may includeas an example 33 directional modes and at least two non-directionalmodes. The non-directional modes may include DC mode and planar mode.The predictor 110 may determine the prediction mode to be applied to thecurrent block by using the prediction mode applied to the neighboringblock.

In case of the inter-prediction, the predictor 110 may derive theprediction sample for the current block based on a sample specified by amotion vector on a reference picture. The predictor 110 may derive theprediction sample for the current block by applying any one of a skipmode, a merge mode, and a motion vector prediction (MVP) mode. In caseof the skip mode and the merge mode, the predictor 110 may use motioninformation of the neighboring block as motion information of thecurrent block. In case of the skip mode, unlike in the merge mode, adifference (residual) between the prediction sample and an originalsample is not transmitted. In case of the MVP mode, a motion vector ofthe neighboring block is used as a motion vector predictor and thus isused as a motion vector predictor of the current block to derive amotion vector of the current block.

In case of the inter-prediction, the neighboring block may include aspatial neighboring block existing in the current picture and a temporalneighboring block existing in the reference picture. The referencepicture including the temporal neighboring block may also be called acollocated picture (colPic). Motion information may include the motionvector and a reference picture index. Information such as predictionmode information and motion information may be (entropy) encoded, andthen output as a form of a bitstream.

When motion information of a temporal neighboring block is used in theskip mode and the merge mode, a highest picture in a reference picturelist may be used as a reference picture. Reference pictures included inthe reference picture list may be aligned based on a picture order count(POC) difference between a current picture and a corresponding referencepicture. A POC corresponds to a display order and can be discriminatedfrom a coding order.

The subtractor 115 generates a residual sample which is a differencebetween an original sample and a prediction sample when the skip mode isapplied, the residual sample may not be generated as described above.

The transformer 120 transforms residual samples in units of a transformblock to generate a transform coefficient. The transformer 120 mayperform transformation based on the size of a corresponding transformblock and a prediction mode applied to a coding block or predictionblock spatially overlapping with the transform block. For example,residual samples can be transformed using discrete sine transform (DST)when intra-prediction is applied to the coding block or the predictionblock overlapping with the transform block and the transform block is a4×4 residual array and is transformed using discrete cosine transform(DCT) in other cases.

The quantizer 125 may quantize the transform coefficients to generate aquantized transform coefficient.

The re-arranger 130 rearranges quantized transform coefficients. There-arranger 130 may rearrange the quantized transform coefficients inthe form of a block into a one-dimensional vector through a coefficientscanning method. Although the re-arranger 130 is described as a separatecomponent, the re-arranger 130 may be a part of the quantizer 125.

The entropy encoder 135 may perform entropy-encoding on the quantizedtransform coefficients. The entropy encoding may include an encodingmethod, for example, an exponential Golomb, a context-adaptive variablelength coding (CAVLC), a context-adaptive binary arithmetic coding(CABAC), or the like. The entropy encoder 135 may perform encodingtogether or separately on information (e.g., a syntax element value orthe like) required for video reconstruction in addition to the quantizedtransform coefficients. The entropy-encoded information may betransmitted or stored in unit of a network abstraction layer (NAL) in abitstream form.

The dequantizer 140 dequantizes values (transform coefficients)quantized by the quantizer 125 and the inverse transformer 145 inverselytransforms values dequantized by the dequantizer 135 to generate aresidual sample.

The adder 150 adds a residual sample to a prediction sample toreconstruct a picture. The residual sample may be added to theprediction sample in units of a block to generate a reconstructed block.Although the adder 150 is described as a separate component, the adder150 may be a part of the predictor 110.

The filter 155 may apply deblocking filtering and/or a sample adaptiveoffset to the reconstructed picture. Artifacts at a block boundary inthe reconstructed picture or distortion in quantization can be correctedthrough deblocking filtering and/or sample adaptive offset. Sampleadaptive offset may be applied in units of a sample after deblockingfiltering is completed. The filter 155 may apply an adaptive loop filter(ALF) to the reconstructed picture. The ALF may be applied to thereconstructed picture to which deblocking filtering and/or sampleadaptive offset has been applied.

The memory 160 may store a reconstructed picture or informationnecessary for encoding/decoding. Here, the reconstructed picture may bethe reconstructed picture filtered by the filter 155. The storedreconstructed picture may be used as a reference picture for (inter)prediction of other pictures. For example, the memory 160 may store(reference) pictures used for inter-prediction. Here, pictures used forinter-prediction may be designated according to a reference picture setor a reference picture list.

FIG. 2 briefly illustrates a structure of a video decoding device towhich the present disclosure is applicable.

Referring to FIG. 2, a video decoding device 200 includes an entropydecoder 210, a re-arranger 220, a dequantizer 230, an inversetransformer 240, a predictor 250, an adder 260, a filter 270, and amemory 280.

When a bitstream including video information is input, the videodecoding device 200 may reconstruct a video in association with aprocedure by which video information is procedured in the video encodingdevice.

For example, the video decoding device 200 may perform video decoding byusing a procedureing unit applied in the video encoding device.Therefore, the procedureing unit block of video decoding may be a codingunit block, a prediction unit block, or a transform unit block. As aunit block of decoding, the coding unit block may be split according toa quad tree structure from a largest coding unit block. As a blockpartitioned from the coding unit block, the prediction unit block may bea unit block of sample prediction. In this case, the prediction unitblock may be divided into sub blocks. As a coding unit block, thetransform unit block may be split according to the quad tree structure,and may be a unit block for deriving a transform coefficient or a unitblock for deriving a residual signal from the transform coefficient.

The entropy decoder 210 may parse the bitstream to output informationrequired for video reconstruction or picture reconstruction. Forexample, the entropy decoder 210 may decode information in the bitstreambased on a coding method such as exponential Golomb encoding, CAVLC,CABAC, or the like, and may output a value of a syntax element requiredfor video reconstruction and a quantized value of a transformcoefficient regarding a residual.

More specifically, a CABAC entropy decoding method can receive a bincorresponding to each syntax element in a bitstream, determine a contextmodel using decoding target syntax element information and decodinginformation of neighboring and decoding target blocks or information ofamabol/bin decoded in a previous step, predict bin generationprobability according to the determined context model and performarithmetic decoding of the bin to generate a symbol corresponding toeach syntax element value. Here, the CABAC entropy decoding method canupdate the context model using information of a symbol/bin decoded for acontext model of the next symbol/bin after determination of the contextmodel.

Information about prediction among information decoded in the entropydecoder 210 may be provided to the predictor 250 and residual values,that is, quantized transform coefficients, on which entropy decoding hasbeen performed by the entropy decoder 210 may be input to there-arranger 220.

The re-arranger 220 may rearrange the quantized transform coefficientsinto a two-dimensional block form. The re-arranger 220 may performrearrangement corresponding to coefficient scanning performed by theencoding device. Although the re-arranger 220 is described as a separatecomponent, the re-arranger 220 may be a part of the quantizer 230.

The dequantizer 230 may de-quantize the quantized transform coefficientsbased on a (de)quantization parameter to output a transform coefficient.In this case, information for deriving a quantization parameter may besignaled from the encoding device.

The inverse transformer 240 may inverse-transform the transformcoefficients to derive residual samples.

The predictor 250 may perform prediction on a current block, and maygenerate a prediction block including prediction samples for the currentblock. A unit of prediction performed in the predictor 250 may be acoding block or may be a transform block or may be a prediction block.

The predictor 250 may determine whether to apply intra-prediction orinter-prediction based on information on a prediction. In this case, aunit for determining which one will be used between the intra-predictionand the inter-prediction may be different from a unit for generating aprediction sample. In addition, a unit for generating the predictionsample may also be different in the inter-prediction and theintra-prediction. For example, which one will be applied between theinter-prediction and the intra-prediction may be determined in unit ofCU. Further, for example, in the inter-prediction, the prediction samplemay be generated by determining the prediction mode in unit of PU, andin the intra-prediction, the prediction sample may be generated in unitof TU by determining the prediction mode in unit of PU.

In case of the intra-prediction, the predictor 250 may derive aprediction sample for a current block based on a neighboring referencesample in a current picture. The predictor 250 may derive the predictionsample for the current block by applying a directional mode or anon-directional mode based on the neighboring reference sample of thecurrent block. In this case, a prediction mode to be applied to thecurrent block may be determined by using an intra-prediction mode of aneighboring block.

In the case of inter-prediction, the predictor 250 may derive aprediction sample for a current block based on a sample specified in areference picture according to a motion vector. The predictor 250 mayderive the prediction sample for the current block using one of the skipmode, the merge mode and the MVP mode. Here, motion information requiredfor inter-prediction of the current block provided by the video encodingdevice, for example, a motion vector and information about a referencepicture index may be acquired or derived based on the information aboutprediction.

In the skip mode and the merge mode, motion information of a neighboringblock may be used as motion information of the current block. Here, theneighboring block may include a spatial neighboring block and a temporalneighboring block.

The predictor 250 may construct a merge candidate list using motioninformation of available neighboring blocks and use informationindicated by a merge index on the merge candidate list as a motionvector of the current block. The merge index may be signaled by theencoding device. Motion information may include a motion vector and areference picture. When motion information of a temporal neighboringblock is used in the skip mode and the merge mode, a highest picture ina reference picture list may be used as a reference picture.

In the case of the skip mode, a difference (residual) between aprediction sample and an original sample is not transmitted,distinguished from the merge mode.

In the case of the MVP mode, the motion vector of the current block maybe derived using a motion vector of a neighboring block as a motionvector predictor. Here, the neighboring block may include a spatialneighboring block and a temporal neighboring block.

When the merge mode is applied, for example, a merge candidate list canbe generated using a motion vector of a reconstructed spatialneighboring block and/or a motion vector corresponding to a Col blockwhich is a temporal neighboring block. A motion vector of a candidateblock selected from the merge candidate list is used as the motionvector of the current block in the merge mode. The aforementionedinformation about prediction may include a merge index indicating acandidate block having the best motion vector selected from candidateblocks included in the merge candidate list. Here, the predictor 250 mayderive the motion vector of the current block using the merge index.

When the MVP (Motion vector Prediction) mode is applied as anotherexample, a motion vector predictor candidate list may be generated usinga motion vector of a reconstructed spatial neighboring block and/or amotion vector corresponding to a Col block which is a temporalneighboring block. That is, the motion vector of the reconstructedspatial neighboring block and/or the motion vector corresponding to theCol block which is the temporal neighboring block may be used as motionvector candidates. The aforementioned information about prediction mayinclude a prediction motion vector index indicating the best motionvector selected from motion vector candidates included in the list.Here, the predictor 250 may select a prediction motion vector of thecurrent block from the motion vector candidates included in the motionvector candidate list using the motion vector index. The predictor ofthe encoding device may obtain a motion vector difference (MVD) betweenthe motion vector of the current block and a motion vector predictor,encode the MVD and output the encoded MVD in the form of a bitstream.That is, the MVD can be obtained by subtracting the motion vectorpredictor from the motion vector of the current block. Here, thepredictor 250 may acquire a motion vector included in the informationabout prediction and derive the motion vector of the current block byadding the motion vector difference to the motion vector predictor. Inaddition, the predictor may obtain or derive a reference picture indexindicating a reference picture from the aforementioned information aboutprediction.

The adder 260 can add a residual sample to a prediction sample toreconstruct a current block or a current picture. The adder 260 mayreconstruct the current picture by adding the residual sample to theprediction sample in units of a block. When the skip mode is applied, aresidual is not transmitted and thus the prediction sample may become areconstructed sample. Although the adder 260 is described as a separatecomponent, the adder 260 may be a part of the predictor 250.

The filter 270 may apply deblocking filtering, sample adaptive offsetand/or ALF to the reconstructed picture. Here, sample adaptive offsetmay be applied in units of a sample after deblocking filtering. The ALFmay be applied after deblocking filtering and/or application of sampleadaptive offset.

The memory 280 may store a reconstructed picture or informationnecessary for decoding. Here, the reconstructed picture may be thereconstructed picture filtered by the filter 270. For example, thememory 280 may store pictures used for inter-prediction. Here, thepictures used for inter-prediction may be designated according to areference picture set or a reference picture list. A reconstructedpicture may be used as a reference picture for other pictures. Thememory 280 may output reconstructed pictures in an output order.

As described above, in the case that an inter prediction is performedfor a target block, motion information for the target block may begenerated by applying a skip mode, a merge mode, or an adaptive motionvector prediction (AMVP) mode and encoded and outputted. In this case,in the motion information for the target block, a distortion may becalculated and included owing to the procedure of being encoded in theunit of block, and through this, the motion information that representsa reconstructed block of the target block may not be reflectedperfectly. Particularly, in the case that the merge mode is applied tothe target block, an accuracy of the motion information for the targetblock may be degraded. That is, there is big difference between aprediction block derived through the motion information for the targetblock and a reconstructed block of the target block. In this case, themotion information for the target block is used in a decoding procedureof a next block neighboring the target block, and a distortion may bepropagated, and through this, overall coding efficiency may be degraded.

Accordingly, in the present invention, a method is proposed thatmodified motion information for the target block is calculated based ona derived reconstructed block after the decoding procedure of the targetblock, the motion information of the target block is updated based onthe modified motion information such that a next block neighboring thetarget block may derive more accurate motion information. Through this,overall coding efficiency may be improved.

FIG. 3 illustrates an example of a case of performing an interprediction based on uni-directional motion information and a case ofperforming an inter prediction based on bi-directional motioninformation which is applied to a target block. The bi-directionalmotion information may include L0 reference picture index and L0 motionvector, and L1 reference picture index and L1 motion vector, and theuni-directional motion information may include L0 reference pictureindex and L0 motion vector or L1 reference picture index and L1 motionvector. The L0 indicates reference picture list L0 (list 0) and the L1indicates reference picture list L1 (list 1). In the image codingprocedure, a method used for the inter prediction may include derivationof the motion information by a motion estimation and a motioncompensation. As shown in FIG. 3, the motion estimation may be indicatedin the procedure of deriving a block matched to the target blocktargeted to reference pictures which is encoded before encoding of thetarget picture in which the target block is included. The block matchedto the target block may be defined as a reference block, and a positiondifference between the target block derived by assuming that thereference block is included in the target picture which is the same asthe target block and the reference block may be defined as a motionvector of the target block. The motion compensation may include auni-directional method in which one reference block is derived and usedand a bi-directional method in which two reference blocks are derivedand used.

The information including information for the motion vector for thetarget block and the reference picture may be defined as motioninformation of the target block. The information for the referencepicture may include a reference picture list and a reference pictureindex that indicates a reference picture included in the referencepicture list. An encoding apparatus may store the motion information forthe target block for a next block neighboring the target block to beencoded next to the target block after the target block is encoded orpictures to be encoded next to the target picture. The stored motioninformation may be used in a method for representing the motioninformation of the next block to be encoded next to the target block orthe pictures to be encoded nest to the target block. The method mayinclude the merge mode which is a method of indexing the motioninformation of the target block and transmitting the motion informationof the next block neighboring the target block as an index and the AMVPmode which is a method of representing the motion vector of the nextblock neighboring the target block only with a differential between themotion vector of the next block and the motion vector of the targetblock.

The present invention proposes a method for updating the motioninformation for the target picture or the target block after theencoding procedure of the target picture or the target block. When aninter prediction is performed and encoded for the target block, themotion information used in the inter prediction may be stored. However,the motion information may include a distortion occurred during theprocedure of being calculated through the block matching method, andsince the motion information may be a value selected through therate-distortion (RD) optimization procedure for the target block, it maybe difficult in reflecting an actual motion of the target blockperfectly. Since the motion information of the target block may used notonly in the inter prediction procedure of the target block but alsoinfluences on the picture which is encoded after encoding of the targetpicture including the next block neighboring the target block encodedafter encoding of the target block and the target picture including thenext block and the target block, overall coding efficiency may bedegraded.

FIG. 4 illustrates an example of an encoding procedure including amethod of updating the motion information of a target block. An encodingapparatus encodes a target block (step, S400). The encoding apparatusmay derive a prediction sample by performing an inter prediction for thetarget block and generate a reconstructed block of the target blockbased on the prediction sample. The encoding apparatus may derive themotion information of the target block for performing the interprediction and generate the information for the inter prediction of thetarget block including the motion information. The motion informationmay be referred to as first motion information.

After performing the encoding procedure of the target block, theencoding apparatus calculates motion information modified based on thereconstructed block of the target block (step, S410). The encodingapparatus may calculate the modified motion information of the targetblock through various methods. The modified motion information may bereferred to as second motion information. At least one method includingdirect methods such as an optical flow (OF) method, a block-matchingmethod, a frequency domain method, and the like and indirect methodssuch as a singularity matching method, a method using a statisticalproperty, and the like may be applied to the method. In addition, thedirect methods and the indirect methods may be applied simultaneously.The detailed contents for the block matching method and the OF methodwill be described below.

The encoding apparatus may be difficult in calculating more accuratemotion information only with the encoded information of the target blockdue to the characteristics of the motion information calculation of thetarget block, and accordingly, the time when the encoding apparatuscalculates the modified motion information may be after performing theencoding procedure of the target picture in which the target block isincluded, not after the encoding procedure of the target block, forexample. That is, the encoding apparatus may perform an encodingprocedure of the target picture and calculate the modified motioninformation based on more information than that of right afterperforming the encoding procedure of the target block.

Meanwhile, the encoding apparatus may further generate motion vectorupdate differential information that indicates differential between theexisting motion vector included in the (first) motion information andthe modified motion information and encode and output it. The motionvector update differential information may be transmitted in the unit ofPU.

The encoding apparatus determines whether to update the motioninformation of the target block (step, S420). The encoding apparatus maydetermine whether to update the (first) motion information through acomparison of accuracy between the (first) motion information and themodified motion information. For example, the encoding apparatus maydetermine whether to update the (first) motion information by using anamount of difference between an image derived by the motion compensationusing each motion information and an original image. In other words, theencoding apparatus may determine whether to update by comparing anamount of data of the residual signal between a reference block derivedbased on each motion information and an original block of the targetblock.

In step S420, in the case that it is determined to update the motioninformation of the target block, the encoding apparatus updates themotion information of the target block based on the modified motioninformation and store the modified motion information (step, S430). Forexample, in the case that the amount of data of the residual signalbetween a specific reference block derived based on the modified motioninformation and the original block is smaller than the amount of data ofthe residual signal between the reference block derived based on the(first) motion information and the original block, the encodingapparatus may update the (first) motion information based on themodified motion information. In this case, the encoding apparatus mayupdate the motion information of the target block by substituting the(first) motion information to the modified motion information and storethe updated motion information including only the modified motioninformation. In addition, the encoding apparatus may update the motioninformation of the target block by adding the modified motioninformation to the (first) motion information and store the updatedmotion information including the (first) motion information and themodified motion information.

Meanwhile, in step S420, in the case that it is not determined to updatethe motion information of the target block, the encoding apparatusstores the (first) motion information (step, S440). For example, in thecase that the amount of data of the residual signal between a specificreference block derived based on the modified motion information and theoriginal block is not smaller than the amount of data of the residualsignal between the reference block derived based on the (first) motioninformation and the original block, the encoding apparatus may store the(first) motion information.

On the other hand, although it is not shown, in the case that themodified motion information is derived without determination of updatethrough comparison between the (first) motion information used in theinter prediction and the modified motion information, the encodingapparatus may update the motion information of the target block based onthe modified motion information.

Meanwhile, since the decoding apparatus is not available to use theoriginal image, the decoding apparatus may receive additionalinformation indicating whether to update the target block. That is, theencoding apparatus may generate and encode the additional informationindicating whether to update and output it through bitstream. Forexample, the additional information indicating whether to update may bereferred as an update flag. The case that the update block is 1 mayindicate that the motion information is updated and the case that theupdate block is 0 may indicate that the motion information is notupdated. For example, the update flag may be transmitted in a unit ofPU. Alternatively, the update flag may be transmitted in a unit of CU, aunit of CTU or a unit of slice and may be transmitted through a higherlevel such as a unit of picture parameter set (PPS) or a unit ofsequence parameter set (SPS).

In addition, the decoding apparatus may determine whether to updatethrough comparison between the (first) motion information of the targetblock and the modified motion information based on the reconstructedblock of the target block without receiving the update flag, and in thecase that it is determined that the motion information of the targetblock is updated, the decoding apparatus may update the motioninformation of the target block based on the modified motion informationand store the updated motion information. In addition, in the case thatthe modified motion information is derived without determination ofupdate through comparison between the motion information used in theinter prediction and the modified motion information, the decodingapparatus may update the motion information of the target block based onthe modified motion information. In this case, the decoding apparatusmay update the motion information of the target block by substitutingthe (first) motion information to the modified motion information andstore the updated motion information including only the modified motioninformation. In addition, the decoding apparatus may update the motioninformation of the target block by adding the modified motioninformation to the (first) motion information and store the updatedmotion information including the (first) motion information and themodified motion information.

In the case that the procedure of updating the motion information isperformed after the encoding procedure of the target block, the encodingprocedure of a next block neighboring the target block of the nextencoding order of the target block may be performed.

FIG. 5 illustrates an example of decoding procedure including a methodof updating the motion information of a target block. The decodingprocedure may be performed in a similar way to the encoding proceduredescribed above. A decoding apparatus decodes a target block (step,S500). In the case that the inter prediction is applied to the targetblock, the decoding apparatus may obtain information for the interprediction of the target block through a bitstream. The decodingapparatus may derive motion information of the target block based on theinformation for the inter prediction and derive a prediction sample byperforming the inter prediction for the target block. The motioninformation may be referred to as first motion information. The decodingapparatus may generate a reconstructed block of the target block basedon the prediction sample.

The decoding apparatus calculates modified motion information of thetarget block (step, S510). The decoding apparatus may calculate themodified motion information through various methods. The modified motioninformation may be referred to as second motion information. At leastone method including direct methods such as an optical flow (OF) method,a block-matching method, a frequency domain method, and the like andindirect methods such as a singularity matching method, a method using astatistical property, and the like may be applied to the method. Inaddition, the direct methods and the indirect methods may be appliedsimultaneously. The detailed contents for the block matching method andthe OF method will be described below.

The decoding apparatus may be difficult in calculating more accuratemotion information only with the decoded information of the target blockdue to the characteristics of the motion information calculation of thetarget block, and accordingly, the time when the decoding apparatuscalculates the modified motion information may be after performing thedecoding procedure of the target picture in which the target block isincluded, not after the decoding procedure of the target block, forexample. That is, the decoding apparatus may perform a decodingprocedure of the target picture and calculate the modified motioninformation based on more information than that of right afterperforming the decoding procedure of the target block.

Meanwhile, the decoding apparatus may also obtain generate motion vectorupdate differential information that indicates differential between theexisting motion vector included in the (first) motion information andthe modified motion information through a bitstream. The motion vectorupdate differential information may be transmitted in the unit of PU. Inthis case, the decoding apparatus may not calculate the modified motioninformation independently through the direct method and the indirectmethod described above but derive the modified motion information bysumming the (first) motion information and the obtained motion vectorupdate differential information. That is, the decoding apparatus mayderive the existing motion vector with a motion vector predictor (MVP)of the modified motion information and by adding the motion vectorupdate differential information to the existing motion vector and derivethe modified motion vector.

The decoding apparatus determines whether to update the motioninformation of the target block (step, S520). The decoding apparatus maydetermine whether to update the (first) motion information through acomparison of accuracy between the (first) motion information and themodified motion information. For example, the decoding apparatus maydetermine whether to update the (first) motion information by using anamount of difference between a reference block derived by the motioncompensation using each motion information and the reconstructed blockof the target block. In other words, the encoding apparatus maydetermine whether to update by comparing an amount of data of theresidual signal between a reference block derived based on each motioninformation and an original block of the target block.

Meanwhile, the decoding apparatus receive additional informationindicating whether to update from the encoding apparatus and determinewhether to update the motion information of the target block based onthe additional information. For example, the additional informationindicating whether to update may be referred as an update flag. The casethat the update block is 1 may indicate that the motion information isupdated and the case that the update block is 0 may indicate that themotion information is not updated. For example, the update flag may betransmitted in a unit of PU.

In step S520, in the case that it is determined to update the motioninformation of the target block, the decoding apparatus updates themotion information of the target block based on the modified motioninformation and store the updated motion information (step, S530). Forexample, in the case that the amount of data of the residual signalbetween a specific reference block derived based on the modified motioninformation and the reconstructed block is smaller than the amount ofdata of the residual signal between the reference block derived based onthe (first) motion information and the reconstructed block, the decodingapparatus may update the (first) motion information based on themodified motion information. In this case, the decoding apparatus mayupdate the motion information of the target block by substituting the(first) motion information to the modified motion information and storethe updated motion information including only the modified motioninformation. In addition, the decoding apparatus may update the motioninformation of the target block by adding the modified motioninformation to the (first) motion information and store the updatedmotion information including the (first) motion information and themodified motion information.

Meanwhile, in step S520, in the case that it is not determined to updatethe motion information of the target block, the decoding apparatusstores the (first) motion information (step, S540). For example, in thecase that the amount of data of the residual signal between a specificreference block derived based on the modified motion information and thereconstructed block is not smaller than the amount of data of theresidual signal between the reference block derived based on the (first)motion information and the reconstructed block, the decoding apparatusmay store the (first) motion information.

On the other hand, although it is not shown, in the case that themodified motion information is derived without determination of updatethrough comparison between the (first) motion information and themodified motion information, the decoding apparatus may update themotion information of the target block based on the modified motioninformation.

Meanwhile, the stored motion information and the motion information fortransmission may have different resolution. In other words, the unit ofmotion vector included in the modified motion information and the unitof motion vector included in the motion information derived based on theinformation for the inter prediction may be different. For example, theunit of motion vector of the motion information derived based on theinformation for the inter prediction may have a unit of ¼ fractionalsample and the unit of motion vector of the modified motion informationcalculated in the decoding apparatus may have a unit of ⅛ fractionalsample or 1/16 fractional sample. The decoding apparatus may adjust theresolution to a resolution required to store in the process ofcalculating the modified motion information or adjust the resolutionthrough a calculation (e.g., round off, multiplication, etc.) in theprocess of storing the modified motion information. In the case that theresolution of the motion vector of the modified motion information ishigher than the motion vector of the motion information derived based onthe information for the inter prediction, an accuracy may be increasedfor a scaling operation for a temporal neighboring motion informationcalculation. In addition, in the case of the decoding apparatus of whichresolutions for transmission of the motion information and for internaloperation are different, there is an effect that decoding may beperformed in accordance with the internal operation criterion.

Meanwhile, in the case that the block matching method is applied amongthe methods for calculating the modified motion information, themodified motion information may be derived as follows. The blockmatching method may be represented as a motion estimation method used inthe encoding apparatus.

A coding apparatus may measure a degree of distortion by accumulatingdifferential values between samples according to phases of a targetblock and a reference block and use these as cost functions, and then,derive a reference block which is the most similar to the target block.That is, the coding apparatus may derive the modified motion informationof the target block based on a reference block of which residual fromthe reconstructed block (or original block) of the target block is aminimum. The reference block of which residual is the minimum may bereferred to as a specific reference block. Sum of absolute differences(SAD) and mean squared error (MSE) may be used for the function ofrepresenting the differential values between samples. The SAD, in whicha degree of distortion is measured by accumulating absolute values ofthe differential values between samples according to phases of thetarget block and the reference block, may be derived based on thefollowing equation.

$\begin{matrix}{{SAD} = {\sum\limits_{j}^{height}{\sum\limits_{i}^{width}{{{{Block}_{cur}\left( {i,j} \right)} - {{Block}_{ref}\left( {i,j} \right)}}}}}} & \left\lbrack {{Equation}\mspace{14mu} 1} \right\rbrack\end{matrix}$

Herein, Block_(cur)(i,j) represents a reconstructed sample (or originalsample) of (i, j) coordinate in the reconstructed block (or originalblock) of the target block, Block_(ref)(i,j) represents a reconstructedsample of (i, j) coordinate in the reference block, width is a width ofthe reconstructed block (or original block) and height represents aheight of the reconstructed block (or original block).

In addition, the MSE, in which a degree of distortion is measured byaccumulating square values of the differential values between samplesaccording to phases of the target block and the reference block, may bederived based on the following equation.

$\begin{matrix}{{MSE} = {\frac{1}{{width}*{height}}{\sum\limits_{j}^{height}{\sum\limits_{i}^{width}\left( {{{Block}_{cur}\left( {i,j} \right)} - {{Block}_{ref}\left( {i,j} \right)}} \right)^{2}}}}} & \left\lbrack {{Equation}\mspace{14mu} 2} \right\rbrack\end{matrix}$

Herein, Block_(cur)(i,j) represents a reconstructed sample (or originalsample) of (i, j) coordinate in the reconstructed block (or originalblock) of the target block, Block_(ref)(i,j) represents a reconstructedsample of (i, j) coordinate in the reference block, width is a width ofthe reconstructed block (or original block) and height represents aheight of the reconstructed block (or original block).

The calculation complexities of the methods for calculating the modifiedmotion information may be changed flexibly depending on a search rangefor searching the specific reference block. Accordingly, in the case ofcalculating the modified motion information of the target block by usingthe block matching method, the decoding apparatus may search only thereference blocks included in a predetermined area from the referenceblock derived through the (first) motion information which is used inthe decoding procedure of the target block, and through this, lowcalculation complexity may be maintained.

FIG. 6 illustrates an example of a method of updating the motioninformation of the target block through the block matching method by thedecoding apparatus. The decoding apparatus decodes the target block(step, S600). In the case that the inter prediction is applied to thetarget block, the decoding apparatus may obtain information for theinter prediction of the target block through a bitstream. Since themethod of updating the motion information of the target block applied tothe encoding apparatus should be applied to the decoding apparatus inthe same manner, the encoding apparatus and the decoding apparatus maycalculate the modified motion information of the target block based onthe reconstructed block of the target block.

The decoding apparatus performs a motion estimation in a referencepicture for the modified motion information by using the reconstructedblock (step, S610). The decoding apparatus may derive a referencepicture for the motion information derived based on the information forthe inter prediction as the specific reference picture for the modifiedmotion information and detect the specific reference picture among thereference blocks in the reference picture. The specific reference blockmay be a reference block of which summation of absolute values of thedifferential values between samples from the reference block among thereference blocks, that is, sum of absolute differences (SAD) is theminimum. In addition, the decoding apparatus may limit the search areafor detecting the specific reference block to a predetermined area fromthe reference block represented through the motion information derivedbased on the information for the inter prediction. In other words, thedecoding apparatus may derive the reference block of which SAD from thereconstructed block is the minimum among the reference blocks located ina predetermined area from the reference block as the specific referenceblock. The decoding apparatus may perform a motion estimation only in apredetermined area from the reference block, and through this, increasereliability of the modified motion information while decreasing thecalculation complexity.

In the case that a size of the reconstructed block of the target blockis greater than a specific size, to calculate the modified motioninformation elaborately, the decoding apparatus partitions thereconstructed block into small blocks of which sizes are smaller thanthe specific size and calculate more detailed motion information basedon each of the small blocks (step, S620). The specific size may bepreconfigured, and the partitioned blocks may be referred to as subreconstructed blocks. The decoding apparatus may partition thereconstructed block into a plurality of sub reconstructed blocks andderive a specific sub reference block in a unit of each subreconstructed block within a reference picture for the modified motioninformation based on the sub reconstructed blocks and derive the derivedspecific sub reference blocks as the specific reference block. Thedecoding apparatus may calculate the modified motion information basedon the specific reference block.

Meanwhile, in the case that the optical flow (OF) method is appliedamong the method for calculating the modified motion information, themodified motion information may be derived as follows. The OF method maybe calculated based on an assumption that a velocity of an object in thetarget block is uniform and sample values of the samples representingthe object is not changed in an image. In the case that the object movesby δx in x axis and δy in y axis for time δt, the following equation maybe established.

I(x, y, t)=I(x+δx, y+δy, t+δt)  [Equation 3]

Herein, I(x,y,t) indicates a sample value of the reconstructed sample in(x, y) position representing the object on time t, which is included inthe target block.

In Equation 3, when the right term is expanded in Taylor series, thefollowing equation may be derived.

$\begin{matrix}{{I\left( {{x + {\delta \; x}},{y + {\delta \; y}},{t + {\delta \; t}}} \right)} \approx {{I\left( {x,y,t} \right)} + {\frac{\partial I}{\partial x}\delta \; x} + {\frac{\partial I}{\partial y}\delta \; y} + {\frac{\partial I}{\partial t}\delta \; t}}} & \left\lbrack {{Equation}\mspace{14mu} 4} \right\rbrack\end{matrix}$

When equation 4 is satisfied, the following equation may be established.

$\begin{matrix}{{{\frac{\partial I}{\partial x}\delta \; x} + {\frac{\partial I}{\partial y}\delta \; y} + {\frac{\partial I}{\partial t}\delta \; t}} = 0} & \left\lbrack {{Equation}\mspace{14mu} 5} \right\rbrack\end{matrix}$

Equation 5 is rewritten, and the following equation may be derived.

$\begin{matrix}{{{\frac{\partial I}{\partial x}\frac{\delta \; x}{\delta \; t}} + {\frac{\partial I}{\partial y}\frac{\delta \; y}{\delta \; t}} + {\frac{\partial I}{\partial t}\frac{\delta \; t}{\delta \; t}}} = 0} & \left\lbrack {{Equation}\mspace{14mu} 6} \right\rbrack \\{{{\frac{\partial I}{\partial x}v_{x}} + {\frac{\partial I}{\partial y}v_{y}} + \frac{\partial I}{\partial t}} = 0} & \;\end{matrix}$

Herein, v_(x) is a vector component in x axis of the calculated motionvector and v_(y) is a vector component in y axis of the calculatedmotion vector. The decoding apparatus may derive values partialderivative values of the object on x axis, y axis and t axis, and derivethe motion vector (v_(x), v_(y)) of the current position, that is, theposition of the reconstructed sample by applying the derived partialderivative values to the equation. In this case, for example, thedecoding apparatus may configure the reconstructed samples in thereconstructed block of the target block representing the object toreconstructed samples included in an area unit of 3×3 size, andcalculate the motion vector (v_(x), v_(y)) which approaches the leftterm to 0 by applying the equation to the reconstructed samples.

The format of storing the modified motion information of the targetblock calculated by the coding apparatus may have various formats, sinceit may be used for the picture coded after the coding procedure of thetarget picture included in a next block neighboring the target block orthe target block, it may be beneficent that the modified motioninformation may be stored in the same format as the motion informationused in the prediction of the target block. The motion information usedin the prediction may include information on whether it isuni-prediction or bi-prediction, a reference picture index and a motionvector. The modified motion information may be calculated to have thesame format as the motion information used in the prediction.

In the case that there is only one reference picture of the targetpicture, that is, the bi-prediction is available be performed except thecase that a value of picture order count (POC) of the target pictureis 1. Generally, the prediction performance may be high in the case thatthe encoding apparatus performs the bi-prediction in comparison with thecase that the encoding apparatus performs the uni-prediction.Accordingly, in the case that the target picture is available to performthe bi-prediction, to improve the overall coding efficiency of an imageby propagating accurate motion information, the coding apparatus maycalculate and update the modified motion information of the target blockwith the bi-prediction through the method described above. However, evenin the case that the target picture is available to perform thebi-prediction, that is, even in the case that a value of POC of thetarget picture is not 1, an occlusion may occur and the case that thebi-prediction is not available to be performed may occur. The occlusionmay be determined to be occurred when the motion is compensated based onthe modified motion information and the sum of the absolute values ofthe differentials of samples of the prediction sample derived by motioncompensation based on the modified motion information and the samples ofthe reconstructed block (or original block) of the target block isgreater than a predetermined threshold value. For example, in the casethat the modified motion information of the target block is thebi-prediction motion information, when the sum of the absolute values ofthe differentials of samples of the prediction sample derived based onthe L0 motion vector included in the modified motion information and thesamples of the reconstructed block (or original block) of the targetblock is greater than a predetermined threshold value, according to theprediction for the L0, it may be determined that the occlusion occurs.In addition, when the sum of the absolute values of the differentials ofsamples of the prediction sample derived based on the L1 motion vectorincluded in the modified motion information and the samples of thereconstructed block (or original block) of the target block is greaterthan a predetermined threshold value, according to the prediction forthe L1, it may be determined that the occlusion occurs. In the case thatthe occlusion occurs by the prediction for either one of L0 and L1, thecoding apparatus may derive the modified motion information as theuni-prediction motion information except the information for theprediction for the list in which the occlusion occurs.

In the case that the occlusion occurs by the prediction for either oneof L0 and L1 or it is available to calculate the modified motioninformation, the method of updating the motion information of the targetblock may be applied by using the motion information of the neighboringblock of the target block, and alternatively, the method of not updatingthe motion information of the target block may be applied based on themodified motion information. In order to store and propagate accuratemotion information, in the case described above, the method of notupdating the motion information of the target block based on themodified motion information may be more proper.

L0 reference picture index and L1 reference picture index of themodified motion information may indicate one of reference pictures inthe reference picture list (the L0 or the L1). The reference pictureindicated by the reference picture index of the modified motioninformation may be referred to as a specific reference picture. A methodof selecting one of several reference pictures in the reference picturelist may be as described below.

For example, the encoding apparatus may select a reference picture whichis the most recently encoded among the reference pictures included inthe reference picture list. That is, the encoding apparatus may generatea reference picture index included in the modified motion informationthat indicates the reference picture which is the most recently encodedamong the reference pictures included in the reference picture list.

In addition, the encoding apparatus may select a reference picture thathas a POC which is the nearest to the picture order count (POC) of thecurrent picture in the reference picture list. That is, the encodingapparatus may generate a reference picture index included in themodified motion information that indicates the reference picture thathas a POC of which absolute value of the difference from the POC of thetarget picture including the target block among the reference picturesincluded in the reference picture list.

Furthermore, the encoding apparatus may select a reference picture whichis belonged to a lower layer in the hierarchical structure in thereference picture list. The reference picture which is belonged to alower layer may be I-slice or a reference picture encoded by applying alow quantization parameter (QP).

In addition, the encoding apparatus may select a reference picture ofwhich reliability of the motion compensation is highest in the referencepicture list. That is, encoding apparatus may derive a specificreference block of the reconstructed block (or original block) of thetarget block among the reference blocks located in the referencepictures included in the reference picture list and generate a referencepicture index included in the modified motion information that indicatesthe reference picture including the specific reference block which isderived.

The methods of generating the reference picture index of the modifiedmotion information described above may be independently applied or themethods may be applied in combination.

The motion vector included in the modified motion information may bederived through at least one method including the OF method, theblock-matching method, the frequency domain method, and the like, and itmay be required to have the motion vector in a unit of at least minimumblock.

FIG. 7 illustrates reference pictures which can be used in the modifiedmotion information.

Referring to FIG. 7, in the case that a target picture is a picture ofwhich POC value is 3, the bi-prediction may be performed for the targetpicture, a picture of which POC value is 2 and a picture of which POCvalue is 4 may be derived as reference pictures of the target picture.L0 of the target picture may include a picture of which POC value is 2and a picture of which POC value is 0, and respective correspondingvalues of L0 reference picture indices may be 0 and 1. In addition, L1of the target picture may include a picture of which POC value is 4 anda picture of which POC value is 8, and respective corresponding valuesof L1 reference picture indices may be 0 and 1. In this case, theencoding apparatus may select specific reference pictures of themodified motion information of the target picture as a reference picturehaving a POC of which absolute value of a difference from the targetpicture among the reference pictures in each reference picture list isthe smallest. Furthermore, the encoding apparatus may derive a motionvector of the modified motion information in a unit of block of 4×4 sizein the target picture. The OF method may be applied to the method ofderiving the motion vector. The motion vector derived based on areference picture of which POC value is 2 may be stored in L0 motioninformation included in the modified motion information and the motionvector derived based on a reference picture of which POC value is 4 maybe stored in L1 motion information included in the modified motioninformation.

In addition, referring to FIG. 7, in the case that a target picture is apicture of which POC value is 8, the target picture may perform theuni-prediction and a picture of which POC value is 0 may be derived as areference picture of the target picture. L0 of the target picture mayinclude a reference picture of which POC value is 0 and a value of theL0 reference picture index corresponding to the reference picture may be0. In this case, the encoding apparatus may select a specific referencepicture of the modified motion information of the target picture as areference picture having a POC of which absolute value of a differencefrom POC of the target picture among the reference pictures included inL0 is the smallest. Furthermore, the encoding apparatus may derive amotion vector of the modified motion information in a unit of block of4×4 size in the current picture. The OF method may be applied to themethod of deriving the motion vector. The block matching method may beapplied to the method of deriving the motion vector, and the motionvector derived based on a reference picture of which POC value is 0 maybe stored in L0 motion information included in the modified motioninformation.

Referring to the embodiments described above, the reference picture ofthe modified motion information of the target picture may be derived toindicate a reference picture having a POC of which absolute value of adifference from POC of the target picture among the reference picturesincluded in the reference picture list is the smallest, and the motionvector of the modified motion information may be calculated in a unit ofblock of 4×4 size. In the case that the target picture is a picture ofwhich POC is 8, the uni-prediction is performed since the target pictureis a picture in which the inter prediction is performed firstly amongthe pictures shown in FIG. 7, but in the that the target picture is apicture of which POC is 3, the bi-prediction is available and themodified motion information of the target picture may be determined tobe bi-prediction motion information. In addition, even in the case thatthe target picture is a picture of which POC is 3, in the case that themodified motion information is calculated and updated in a unit of blockin the target picture, the encoding apparatus may determine a selectionof one motion information format including the bi-prediction motioninformation and the uni-prediction motion information in a unit ofblock.

In addition, different from the embodiment described above, thereference picture index of the modified motion information may begenerated to indicate a reference picture selected by a hierarchicalstructure, not the reference picture having a POC of which absolutevalue of a difference from POC of the target picture among the referencepictures included in the reference picture list is the smallest. Forexample, in the case that a target picture is a picture of which POC is5, L0 reference picture index of the modified motion information of thetarget picture may indicates a picture of which POC value is 0, not apicture of which POC value is 4. On the timing after performing theencoding procedure of the target picture, most of a reference picturewhich is most recently encoded may indicate a reference picture having aPOC of which absolute value of a difference from POC of the targetpicture is the smallest. However, like the case of a picture of whichPOC value is 6, a reference picture which is most recently encoded is areference picture of which POC value is 2, and a picture having a POC ofwhich absolute value of a difference from POC of the target picture isthe smallest is a picture of which POC value is 4, and accordingly, thecase of indicating different reference picture may occur.

In the case that the motion information of the target block is updatedto the modified motion information through the method described above,the inter prediction of a next block neighboring the target block may beperformed by using the modified motion information (the target block maybe referred to as a first block and the next block may be referred to asa second block). The method of utilizing the modified motion informationin the inter prediction mode of the next block may include a method ofindexing and transmitting the modified motion information and a methodof representing the motion vector of the next block as a differentialvalue between the motion vector of the target block and the motionvector of the next block. The method of indexing and transmitting themodified motion information may be a method in the case that the mergemode is applied to the next block among the inter prediction modes, andthe method of representing the motion vector of the next block as adifferential value between the motion vector of the target block and themotion vector of the next block may be a method in the case that theAMVP mode is applied to the next block among the inter prediction modes.

FIG. 8 illustrates an example of a merge candidate list of a next blockneighboring the target blocks in the case that the motion information ofthe target blocks is updated to the modified motion information of thetarget blocks. Referring to FIG. 8, the coding apparatus may configurethe merge candidate list of the next block including the target blocks.The encoding apparatus may transmit a merge index indicating a blockwhich is the most similar to the motion information of the next blockamong the blocks included in the merge candidate list. In this case, theupdated motion information of the target blocks may be used for motioninformation of a spatial neighboring candidate block of the next block,or alternatively, used for the motion information of a temporalneighboring candidate block. The motion information of the target blocksmay include the modified motion information of the target blocks. Asshown in FIG. 8, the motion information of the spatial neighboringcandidate block may represent one motion information among the storedmotion information of the blocks neighboring the next block on A1, B1,B0, A0 and B2 positions. The motion information of the target blocksencoded before the encoding of the next block may be updated, and theupdated motion information may influence on the encoding procedure ofthe next block. In the case that motion information including an erroris propagated to the next block through the merge mode, the error may beaccumulated and propagated, but the propagated error may be reduced byupdating the motion information in each block through the methoddescribed above.

FIG. 9 illustrates an example of a merge candidate list of a next blockneighboring the target blocks in the case that the motion information ofthe target blocks is updated to the modified motion information of thetarget blocks. The (first) motion information and the modified motioninformation used in prediction of the target block neighboring the nextblock may be stored. In this case, the coding apparatus may configurethe target block indicating the modified motion information of thetarget block which is newly calculated in the merge candidate list ofthe next block as a separate merge candidate block. As shown in FIG. 9,the target block indicating the modified motion information of thetarget block may be inserted to the merge candidate list next to thetarget block indicating the (first) motion information which is appliedin the encoding procedure of the target block neighboring the nextblock. In addition, the priority of the merge candidate list may bechanged, and the number of inserted positions may be differentlyconfigured.

Although A1 and B1 are illustrated as an example of the target blocks inwhich the modified motion information is derived, this is just anexample, and the modified motion information may be derived even for theremaining A0, B0, B2, T0 and T1, and the merge candidate list may beconfigured based on it.

For example, the next block may be located on a picture different fromthe target block, and the temporal neighboring candidate block of thenext block may be included in the merge candidate list. The motioninformation of the temporal neighboring candidate block may be selectedthrough the merge index of the next block. In this case, the motioninformation of the temporal neighboring candidate block may be updatedto the modified motion information through the method described above,and the temporal neighboring candidate block indicating the updatedmotion information may be inserted in the merge candidate list of thenext block. In addition, through the method described above, the (first)motion information and the modified motion information of the temporalneighboring candidate block may be separately stored. In this case, inthe procedure of generating the merge candidate list of the next block,the temporal neighboring candidate block indicating the modified motioninformation may be inserted as an additional candidate of the mergecandidate list.

Different from the contents shown in FIG. 9, the AMVP mode may beapplied to the next block. In the case that the AMVP mode is applied tothe next block, the coding apparatus may generate a motion vectorpredictor candidate list based on the motion information of theneighboring blocks of the next block.

FIG. 10 illustrates an example of a motion vector predictor candidatelist of a next block neighboring the target block in the case that themotion information of the target block is updated based on the modifiedmotion information of the target block. The encoding apparatus mayselect the motion information according to a specific condition amongthe motion information included in the motion vector predictor candidatelist and transmit a motion vector difference (MVD) value from an indexindicating the selected motion information, the motion vector of theselected motion information motion vector of the motion information ofthe next block. Like the method of inserting the block indicating theupdated motion information of the neighboring next block in the mergecandidate list in spatial or temporal merge candidate block in the mergemode described above, the procedure of generating a list including themotion information of the neighboring blocks of the AMVP mode may beapplied to the method of inserting the updated motion information of theneighboring block as a spatial or temporal motion vector predictor (MVP)candidate. As shown in FIG. 10, the coding apparatus may detect whetherthe updated motion information of target block A0 and target block A1 ofthe neighboring next block is in accordance with a predefined specificcondition in an order of the arrow direction and derive the updatedmotion information in accordance with the specific condition which isdetected firstly as MVP A which is included in the motion predictorcandidate list.

In addition, the coding apparatus may detect whether the updated motioninformation of target block B1, target block B1 and target block B2 ofthe neighboring next block is in accordance with a predefined specificcondition in an order of the arrow direction and derive the updatedmotion information in accordance with the specific condition which isdetected firstly as MVP B which is included in the motion predictorcandidate list.

Furthermore, the next block may be positioned in different picture fromthe target block, and the temporal MVP candidate of the next block maybe included in the motion vector predictor candidate list. As shown inFIG. 10, the coding apparatus may detect whether it is in accordancewith a predefined specific condition in an order of the updated motioninformation of target block T0 located in the reference picture of thenext block to the updated motion information of target block T1 andderive the updated motion information in accordance with the specificcondition which is detected firstly as MVP Col which is included in themotion predictor candidate list. In the case that the MVP A, the MVP Band/or the MVP Col are the updated motion information by being replaceby the modified motion information, the updated motion information maybe used for the motion vector predictor candidate of the next block.

Meanwhile, in the case that smaller number of the MVP candidates in themotion predictor candidate list are derived than a specific number, thecoding apparatus derives zero vector as MVP zero and include it in themotion predictor candidate list.

FIG. 11 illustrates an example of a motion vector predictor candidatelist of a next block neighboring the target block in the case that themotion information of the target block is updated based on the modifiedmotion information of the target block. In the case that the motioninformation of the target block is the updated motion informationincluding the modified motion information and the existing motioninformation, the coding apparatus may detect whether the existing motioninformation of target block A0 and target block A1 of the neighboringnext block is in accordance with a specific condition in an order of thearrow direction and derive the existing motion information in accordancewith the specific condition which is detected firstly as MVP A which isincluded in the motion predictor candidate list.

In addition, the coding apparatus may detect whether the modified motioninformation of the target block A0 and the target block A1 is inaccordance with a specific condition in an order of the arrow directionand derive the modified motion information in accordance with thespecific condition which is detected firstly as updated MVP A which isincluded in the motion predictor candidate list.

Furthermore, the coding apparatus may detect whether the existing motioninformation of target block B1, target block B1 and target block B2 ofthe neighboring next block is in accordance with a predefined specificcondition in an order of the arrow direction and derive the existingmotion information in accordance with the specific condition which isdetected firstly as MVP B which is included in the motion predictorcandidate list.

In addition, the coding apparatus may detect whether the modified motioninformation of target block B1, target block B1 and target block B2 isin accordance with a specific condition in an order of the arrowdirection and derive the updated motion information in accordance withthe specific condition which is detected firstly as updated MVP B whichis included in the motion predictor candidate list.

Furthermore, the coding apparatus may detect whether it is in accordancewith a specific condition in an order of the existing motion informationof target block T0 located in the reference picture of the next block tothe existing motion information of target block T1 and derive theexisting motion information in accordance with the specific conditionwhich is detected firstly as MVP Col which is included in the motionpredictor candidate list.

In addition, the coding apparatus may detect whether it is in accordancewith a specific condition in an order of the modified motion informationof target block T0 located in the reference picture of the next block tothe modified motion information of target block T1 and derive themodified motion information in accordance with the specific conditionwhich is detected firstly as MVP Col which is included in the motionpredictor candidate list.

Meanwhile, in the case that smaller number of the MVP candidates in themotion predictor candidate list are derived than a specific number, thecoding apparatus derives zero vector as MVP zero and include it in themotion predictor candidate list.

FIG. 12 schematically illustrates a video encoding method by an encodingapparatus according to the present invention. The method shown in FIG.12 may be performed by the encoding apparatus shown in FIG. 1. In adetailed example, steps S1200 to S1230 of FIG. 12 may be performed by aprediction unit of the encoding apparatus and step S1240 may beperformed by a memory of the encoding apparatus.

The encoding apparatus generates the motion information for the targetblock (step, S1200). The encoding apparatus may apply the interprediction to the target block. In the case that the inter prediction isapplied to the target block, the encoding apparatus may generate themotion information for the target block by applying at least one of theskip mode, the merge mode and the adaptive motion vector prediction(AMVP) mode. The motion information may be referred to as first motioninformation. In the case of the skip mode and the merge mode, theencoding apparatus may generate the motion information for the targetblock based on the motion information of a neighboring block of thetarget block. The motion information may include a motion vector and areference picture index. The motion information may be bi-predictionmotion information or uni-prediction motion information. Thebi-prediction motion information may include L0 reference picture indexand L0 motion vector, and L1 reference picture index and L1 motionvector, and the uni-directional motion information may include L0reference picture index and L0 motion vector or L1 reference pictureindex and L1 motion vector. The L0 indicates reference picture list L0(list 0) and the L1 indicates reference picture list L1 (list 1).

In the case of the AMVP mode, the encoding apparatus may derive themotion vector of the target block by using the motion vector of theneighboring block of the target block as a motion vector predictor (MVP)and generate the motion information including the motion vector and thereference picture index for the motion vector.

The encoding apparatus derives a prediction sample by performing theinter prediction for the target block based on the motion information(step, S1210). The encoding apparatus may generate the prediction sampleof the target block based on the reference picture index and the motionvector included in the motion information.

The encoding apparatus generates a reconstructed block based on theprediction sample (step, S1220). The encoding apparatus may generate thereconstructed block of the target block based on the prediction sampleor generate a residual signal of the target block and generate thereconstructed block of the target block based on the residual signal andthe prediction sample.

The encoding apparatus generates modified motion information for thetarget block based on the reconstructed block (step, S1230). Theencoding apparatus may calculate the modified reference picture indexindicating a specific reference picture for the modified motioninformation and a modified motion vector for the specific referencepicture through various methods. The modified motion information may bereferred to as second motion information. At least one method includingdirect methods such as an optical flow (OF) method, a block-matchingmethod, a frequency domain method, and the like and indirect methodssuch as a singularity matching method, a method using a statisticalproperty, and the like may be applied to the method. In addition, thedirect methods and the indirect methods may be applied simultaneously.

For example, the encoding apparatus may generate the modified motioninformation through the block matching method. In this case, theencoding apparatus may measure a degree of distortion by accumulatingdifferential values between samples according to phases of thereconstructed block of the target block and the reference block and usethese as cost functions, and then, detect a specific reference block ofthe reconstructed block. The encoding apparatus may generate themodified motion information based on the detected specific referenceblock. In other words, the encoding apparatus may generate the modifiedmotion information including the modified reference picture indexindicating the specific reference index and the modified motion vectorindicating the specific reference block in the specific referencepicture. The encoding apparatus may detect the reference block of whichsummation of the absolute values (or square values) of the differentialsbetween samples according to the phase of the reconstructed block of thetarget block among the reference blocks in the specific referencepicture as the specific reference block and derive the modified motioninformation based on the specific reference block. As a method forrepresenting the summation of the absolute values of the differentials,the sum of absolute differences (SAD) may be used. In this case, thesummation of the absolute values of the differentials may be calculatedby using Equation 1 described above. In addition, as a method forrepresenting the summation of the absolute values of the differentials,the mean squared error (MSE) may be used. In this case, the summation ofthe absolute values of the differentials may be calculated by usingEquation 2 described above.

In addition, the specific reference picture for the modified motioninformation may be derived as the reference picture indicated by thereference picture index included in the (first) motion information, andthe search area for detecting the specific reference block may belimited to the reference blocks located in a predetermined area from thereference block derived in the reference picture based on the motionvector related to the reference picture included in the (first) motioninformation. That is, the encoding apparatus may derive the referenceblock of which SAD from the reconstructed block is the minimum among thereference blocks located in a predetermined area from the referenceblock derived in the reference picture as the specific reference blockbased on the motion vector related to the reference picture included inthe (first) motion information.

In addition, in the case that a size of the reconstructed block isgreater than a predetermined size, the reconstructed block may bepartitioned into a plurality of sub reconstructed blocks, and a specificsub reconstructed block may be derived in a unit of sub reconstructedblock in the specific reference picture. In this case, the encodingapparatus may derive the specific reference block based on the derivedspecific sub reconstructed blocks.

As another example, the encoding apparatus may generate the modifiedmotion information through the OF method. In this case, the encodingapparatus may calculate the motion vector for the modified motioninformation of the target block based on an assumption that a velocityof an object in the target block is uniform and sample values of thesamples representing the object is not changed in an image. The motionvector may be calculated through Equation 6 described above. The area ofthe samples indicating the object included in the target block may beconfigured as an area of 3×3 size.

In the case of calculating the modified motion information through themethods described above, the encoding apparatus may calculate themodified motion information so as to have the same format as the (first)motion information. That is, the encoding apparatus may calculate themodified motion information so as to have the same format as the motioninformation between the bi-prediction motion information and theuni-prediction motion information. The bi-prediction motion informationmay include L0 reference picture index and L0 motion vector, and L1reference picture index and L1 motion vector, and the uni-directionalmotion information may include L0 reference picture index and L0 motionvector or L1 reference picture index and L1 motion vector. The L0indicates reference picture list L0 (list 0) and the L1 indicatesreference picture list L1 (list 1).

In addition, the bi-prediction is available to be performed in thetarget picture in which the target block is included, the encodingapparatus may calculate the modified motion information as thebi-prediction motion information through the method described above.However, after the modified motion information is calculated as thebi-prediction motion information, in the case that an occlusion occurson one of the specific reference blocks derived by the bi-predictionmotion information, the encoding apparatus may derive the modifiedmotion information as the uni-prediction motion information except themotion information for the reference picture list of the specificreference picture in which the specific reference block is included inwhich the occlusion occurs. Whether the occlusion is occurred may bedetermined to be occurred when the differential value between thesamples according to the phase of the reference block derived based onthe modified motion information and the reconstructed block of thetarget block is greater than a specific threshold value. The thresholdvalue may be preconfigured.

For example, the encoding apparatus calculates the modified motioninformation as the bi-prediction motion information, and in the casethat the differential value between the samples according to the phaseof the specific reference block derived based on L0 motion vector and L0reference picture index of the modified motion information and thereconstructed block of the target block is greater than thepreconfigured threshold value, the encoding apparatus may derive themodified motion information as the uni-prediction motion informationincluding L1 motion vector and L1 reference picture index.

For another example, the encoding apparatus calculates the modifiedmotion information as the bi-prediction motion information, and in thecase that the differential value between the samples according to thephase of the specific reference block derived based on L1 motion vectorand L1 reference picture index of the modified motion information andthe reconstructed block of the target block is greater than thepreconfigured threshold value, the encoding apparatus may derive themodified motion information as the uni-prediction motion informationincluding L0 motion vector and L0 reference picture index.

In addition, in the case that the differential value between the samplesaccording to the phase of the specific reference block derived based onL0 motion vector and L0 reference picture index and the reconstructedblock of the target block is greater than the preconfigured thresholdvalue and in the case that the differential value between the samplesaccording to the phase of the specific reference block derived based onL1 motion vector and L1 reference picture index and the reconstructedblock of the target block is greater than the preconfigured thresholdvalue, the encoding apparatus may derive the motion information of theneighboring block of the target block as the modified motion informationor may not calculate the modified motion information.

Meanwhile, the encoding apparatus may select the specific referencepicture indicated by the modified reference picture index included inthe modified motion information through various methods.

For example, the encoding apparatus may select the reference picturewhich is the most recently encoded among the reference pictures includedin the reference picture list L0 and generate the modified L0 referencepicture index indicating the reference picture. In addition, theencoding apparatus may select the reference picture which is the mostrecently encoded among the reference pictures included in the referencepicture list L1 and generate the modified L1 reference picture indexindicating the reference picture.

For another example, the encoding apparatus may select the referencepicture having a POC of which an absolute value of a difference from apicture order count (POC) of the target picture among the referencepictures included in the L0 reference picture index and generate themodified L0 reference picture index indicating the reference picture. Inaddition, the encoding apparatus may select the reference picture havinga POC of which an absolute value of a difference from a picture ordercount (POC) of the target picture among the reference pictures includedin the L1 reference picture index and generate the modified L1 referencepicture index indicating the reference picture.

For another example, the encoding apparatus may select the referencepicture that belongs to the lowest layer on the hierarchical structureamong the reference pictures included in each of the reference picturelists and generate the modified reference picture index indicating thereference picture. The reference picture that belongs to the lowestlayer may be I-slice or a reference picture which is encoded by applyinga low quantization parameter (QP).

For another example, the encoding apparatus may select the referencepicture including a reference block of which reliability of the motioncompensation is the highest in the reference picture list and generatethe modified reference picture index indicating the reference picture.In other words, the encoding apparatus may derive the specific referenceblock of the reconstructed block of the target block based on thereference pictures included in the reference picture list and generatethe modified reference picture index indicating the specific referencepicture including the derived specific reference block.

The methods of generating the modified reference picture index of themodified motion information described above may be independently appliedor the methods may be applied in combination.

In addition, although it is not shown, the encoding apparatus maygenerate the modified motion information of the target block based onthe original block of the target block. The encoding apparatus mayderive the specific reference block of the original block among thereference blocks included in the reference pictures and generate themodified motion information indicating the derived specific referenceblock.

The encoding apparatus updates the motion information of the targetblock based on the modified motion information (step, S1240). Theencoding apparatus may store the modified motion information and updatethe motion information of the target block. The encoding apparatus mayupdate the motion information of the target block by substituting themotion information used in predicting the target block to the modifiedmotion information. In addition, the encoding apparatus may update themotion information of the target block by storing all the motioninformation and the modified motion information used in predicting thetarget block. The updated motion information may be used for the motioninformation of the next block neighboring the target block. For example,the merge mode is applied to the next block neighboring the targetblock, the merge candidate list of the next block may include the targetblock. In the case that the motion information of the target block isstored by substituting the motion information used in predicting thetarget block to the modified motion information, the merge candidatelist of the next block may include the target block indicating themodified motion information. In addition, all the motion information andthe modified motion information used in predicting the target block arestored in the motion information of the target block, the mergecandidate list of the next block may include the target block indicatingthe motion information used in predicting the target block and thetarget block indicating the modified motion information. The targetblock indicating the modified motion information may be inserted as thespatial neighboring candidate block in the merge candidate list orinserted as the temporal neighboring candidate block in the mergecandidate list.

For another example, the AMVP mode is applied to the next blockneighboring the target block, like the method of inserting the updatedmotion information of the target block neighboring the next block in themerge candidate list in the merge mode described above as the spatial ortemporal neighboring motion information, the method of inserting theupdated motion information of the neighboring block in the motion vectorpredictor candidate list in the next block above as the spatial ortemporal motion vector predictor candidate may be applied. That is, theencoding apparatus may generate the motion vector predictor candidatelist including the updated motion information of the target blockneighboring the next block.

For example, in the case that the updated motion information of thetarget block neighboring the next block includes only the modifiedmotion information, the motion vector predictor candidate list mayinclude the modified motion information as the spatial motion vectorpredictor candidate.

For another example, in the case that the updated motion information ofthe target block neighboring the next block includes the modified motioninformation and the existing motion information of the target block, themotion vector predictor candidate list may include the modified motioninformation selected according to a specific condition among theexisting motion information selected according to a specific conditionand the modified motion information of the target blocks neighboring thenext block among the existing motion information of the target blockneighboring the next block as the respective spatial motion vectorpredictor candidate.

Meanwhile, the motion vector predictor candidate list of the next blockmay include the updated motion information of the collocated block ofthe same position as the position of the next block and the updatedmotion information of the neighboring block of the same position blockas the temporal motion vector predictor candidate list in the referencepicture of the next block. For example, in the case that the updatedmotion information of the collocated block of the same position as theposition of the next block and the updated motion information of theneighboring block of the same position block include only the modifiedmotion information of each block, the motion vector predictor candidatelist may include the modified motion information as the temporal motionvector predictor candidate list.

For another example, in the case that the updated motion informationincludes the same position block, the respective modified motioninformation of the neighboring block of the same position block and theexisting motion information of the same position block, the motionvector predictor candidate list may include the modified motioninformation selected according to the specific condition among theexisting motion information selected according to the specific conditionand the modified motion information among the existing motioninformation as separate spatial motion vector predictor candidate,respectively.

Meanwhile, the stored motion information and the motion information fortransmission may have different resolution. For example, the encodingapparatus encodes and transmits the information for the motioninformation and the modified motion information is stored for the motioninformation neighboring the target block, the unit of motion vectorincluded in the motion information may have a unit of ¼ fractionalsample and the unit of motion vector of the modified motion informationmay represent a unit of ⅛ fractional sample or 1/16 fractional sample.

Meanwhile, although it is not shown, the encoding apparatus maydetermine whether to update the (first) motion information of the targetblock by performing the comparison procedure between the (first) motioninformation and the modified motion information based on thereconstructed block of the target block. For example, the encodingapparatus may determine whether to update the target block by comparingthe amount of data of the residual signal of the specific referenceblock derived based on the modified motion information and thereconstructed block of the target block with the amount of data of theresidual signal of the reference block derived based on the motioninformation and the reconstructed block. Among the amounts of data, inthe case that the amount of data of the residual signal of the specificreference block and the reconstructed block is smaller, the encodingapparatus may determine to update the motion information and the targetblock. In addition, among the amounts of data, in the case that theamount of data of the residual signal of the specific reference blockand the reconstructed block is not smaller, the encoding apparatus maydetermine not to update the motion information and the target block.

In addition, the encoding apparatus may determine whether to update the(first) motion information of the target block by performing thecomparison procedure between the motion information and the modifiedmotion information based on the original block of the target block. Forexample, the encoding apparatus may determine whether to update thetarget block by comparing the amount of data of the residual signal ofthe specific reference block derived based on the modified motioninformation and the original block of the target block with the amountof data of the residual signal of the reference block derived based onthe motion information and the original block. Among the amounts ofdata, in the case that the amount of data of the residual signal of thespecific reference block and the original block is smaller, the encodingapparatus may determine to update the motion information and the targetblock. In addition, among the amounts of data, in the case that theamount of data of the residual signal of the specific reference blockand the original block is not smaller, the encoding apparatus maydetermine not to update the motion information and the target block.

Meanwhile, the encoding apparatus may generate and encode the additionalinformation indicating whether to update and output it throughbitstream. For example, the additional information indicating whether toupdate may be referred as an update flag. The case that the update blockis 1 may indicate that the motion information is updated and the casethat the update block is 0 may indicate that the motion information isnot updated. For example, the update flag may be transmitted in a unitof PU. Alternatively, the update flag may be transmitted in a unit ofCU, a unit of CTU or a unit of slice and may be transmitted through ahigher level such as a unit of picture parameter set (PPS) or a unit ofsequence parameter set (SPS).

Meanwhile, the encoding apparatus may further generate motion vectorupdate differential information that indicates differential between theexisting motion vector and the modified motion information and encodeand output it. The motion vector update differential information may betransmitted in the unit of PU.

Although it is not shown, the encoding apparatus may encode and outputthe information on the residual sample for the target block. Theinformation on the residual sample may include transform coefficientsfor the residual sample.

FIG. 13 schematically illustrates a video decoding method by a decodingapparatus according to the present invention. The method shown in FIG.13 may be performed by the decoding apparatus shown in FIG. 2. In adetailed example, step S1300 of FIG. 13 may be performed by an entropydecoding unit of the decoding apparatus, steps S1310 to S1340 may beperformed by a prediction unit of the decoding apparatus and step S1350may be performed by a memory of the decoding apparatus.

The decoding apparatus obtains the information for the inter predictionof the target block through a bitstream (step, S1300). The interprediction or an intra prediction may be applied to the target block. Inthe case that the inter prediction is applied to the target block, thedecoding apparatus may obtain the information for the inter predictionof the target block through the bitstream. In addition, the decodingapparatus may obtain the motion vector update differential informationthat indicates differential between the existing motion vector of thetarget block and the modified motion information through the bitstream.Furthermore, the decoding apparatus may obtain addition informationwhether to update the target block through the bitstream. For example,the addition information indicating whether to update may be referred toas an update flag.

The decoding apparatus derives the motion information of the targetblock based on the information on the inter prediction (step, S1310).The motion information may be referred to as first motion information.The information on the inter prediction may represent a mode which isapplied to the target block among the skip mode, the merge mode and theadaptive motion vector prediction (AMVP) mode. In the case that the skipmode or the merge mode is applied to the target block, the decodingapparatus may generate the merge candidate list including neighboringblocks of the target block and obtain a merge index indicating aneighboring block among the neighboring blocks included in the mergecandidate list. The merge index may be included in the information onthe inter prediction. The decoding apparatus may derive the motioninformation of the neighboring block indicated by the merge index as themotion information of the target block.

In the case that the AMVP mode is applied to the target block, thedecoding apparatus may generate a list based on the neighboring blocksof the target block like in the merge mode. The decoding apparatus maygenerate the index indicating a neighboring block among the neighboringblocks included in the generated list and a motion vector difference(MVD) between the motion vector of the neighboring block indicated bythe index and the motion vector of the target block. The index and theMVD may be included in the information on the inter prediction. Thedecoding apparatus may generate the motion information of the targetblock based on the motion vector of the neighboring block indicated bythe index and the MVD.

The motion information may include a motion vector and a referencepicture index. The motion information may be bi-prediction motioninformation or uni-prediction motion information. The bi-predictionmotion information may include L0 reference picture index and L0 motionvector, and L1 reference picture index and L1 motion vector, and theuni-directional motion information may include L0 reference pictureindex and L0 motion vector or L1 reference picture index and L1 motionvector. The L0 indicates reference picture list L0 (list 0) and the L1indicates reference picture list L1 (list 1).

The decoding apparatus derives a prediction sample by performing theinter prediction for the target block based on the motion information(step, S1320). The decoding apparatus may generate the prediction sampleof the target block based on the reference picture index and the motionvector included in the motion information.

The decoding apparatus generates a reconstructed block based on theprediction sample (step, S1330). In the case that the skip mode isapplied to the target block, the decoding apparatus may generate thereconstructed block of the target block based on the prediction sample.In the case that the merge mode or the AMVP mode is applied to thetarget block, the decoding apparatus may generate a residual signal ofthe target block through the bitstream and generate the reconstructedblock of the target block based on the residual signal and theprediction sample.

The encoding apparatus derives modified motion information for thetarget block based on the reconstructed block (step, S1340). Thedecoding apparatus may calculate the modified reference picture indexindicating a specific reference picture for the modified motioninformation and a modified motion vector for the specific referencepicture through various methods. The modified motion information may bereferred to as second motion information. At least one method includingdirect methods such as an optical flow (OF) method, a block-matchingmethod, a frequency domain method, and the like and indirect methodssuch as a singularity matching method, a method using a statisticalproperty, and the like may be applied to the method. In addition, thedirect methods and the indirect methods may be applied simultaneously.

For example, the decoding apparatus may generate the modified motioninformation through the block matching method. In this case, thedecoding apparatus may measure a degree of distortion by accumulatingdifferential values between samples according to phases of thereconstructed block of the target block and the reference block and usethese as cost functions, and then, detect a specific reference block ofthe reconstructed block. The decoding apparatus may generate themodified motion information based on the detected specific referenceblock. In other words, the decoding apparatus may generate the modifiedmotion information including the modified reference picture indexindicating the specific reference index and the modified motion vectorindicating the specific reference block in the specific referencepicture. The decoding apparatus may detect the reference block of whichsummation of the absolute values (or square values) of the differentialsbetween samples according to the phase of the reconstructed block of thetarget block among the reference blocks in the specific referencepicture as the specific reference block and derive the modified motioninformation based on the specific reference block. As a method forrepresenting the summation of the absolute values of the differentials,the sum of absolute differences (SAD) may be used. In this case, thesummation of the absolute values of the differentials may be calculatedby using Equation 1 described above. In addition, as a method forrepresenting the summation of the absolute values of the differentials,the mean squared error (MSE) may be used. In this case, the summation ofthe absolute values of the differentials may be calculated by usingEquation 2 described above.

In addition, the specific reference picture for the modified motioninformation may be derived as the reference picture indicated by thereference picture index included in the (first) motion information, andthe search area for detecting the specific reference block may belimited to the reference blocks located in a predetermined area from thereference block derived in the specific reference picture based on themotion vector related to the reference picture included in the (first)motion information. That is, the decoding apparatus may derive thereference block of which SAD from the reconstructed block is the minimumamong the reference blocks located in a predetermined area from thereference block derived in the specific reference picture as thespecific reference block based on the motion vector related to thereference picture included in the motion information.

In addition, in the case that a size of the reconstructed block isgreater than a predetermined size, the reconstructed block may bepartitioned into a plurality of sub reconstructed blocks, and a specificsub reconstructed block may be derived in a unit of sub reconstructedblock in the specific reference picture. In this case, the decodingapparatus may derive the specific reference block based on the derivedspecific sub reconstructed blocks.

As another example, the decoding apparatus may generate the modifiedmotion information through the OF method. In this case, the decodingapparatus may calculate the modified motion vector for the modifiedmotion information of the target block based on an assumption that avelocity of an object in the target block is uniform and sample valuesof the samples representing the object is not changed in an image. Themotion vector may be calculated through Equation 6 described above. Thearea of the samples indicating the object included in the target blockmay be configured as an area of 3×3 size.

In the case of calculating the modified motion information through themethods described above, the decoding apparatus may calculate themodified motion information so as to have the same format as the (first)motion information. That is, the decoding apparatus may calculate themodified motion information so as to have the same format as the motioninformation between the bi-prediction motion information and theuni-prediction motion information. The bi-prediction motion informationmay include L0 reference picture index and L0 motion vector, and L1reference picture index and L1 motion vector, and the uni-directionalmotion information may include L0 reference picture index and L0 motionvector or L1 reference picture index and L1 motion vector. The L0indicates reference picture list L0 (list 0) and the L1 indicatesreference picture list L1 (list 1).

In addition, the bi-prediction is available to be performed in thetarget picture in which the target block is included, the decodingapparatus may calculate the modified motion information as thebi-prediction motion information through the method described above.However, after the modified motion information is calculated as thebi-prediction motion information, in the case that an occlusion occurson one of the reference blocks derived by the bi-prediction motioninformation, the decoding apparatus may derive the modified motioninformation as the uni-prediction motion information except the motioninformation for the reference picture list of the specific referencepicture in which the specific reference block is included in which theocclusion occurs. Whether the occlusion is occurred may be determined tobe occurred when the differential value between the samples according tothe phase of the specific reference block derived based on the modifiedmotion information and the reconstructed block of the target block isgreater than a specific threshold value. The threshold value may bepreconfigured.

For example, the decoding apparatus calculates the modified motioninformation as the bi-prediction motion information, and in the casethat the differential value between the samples according to the phaseof the specific reference block derived based on L0 motion vector and L0reference picture index of the modified motion information and thereconstructed block of the target block is greater than thepreconfigured threshold value, the encoding apparatus may derive themodified motion information as the uni-prediction motion informationincluding L1 motion vector and L1 reference picture index.

For another example, the decoding apparatus calculates the modifiedmotion information as the bi-prediction motion information, and in thecase that the differential value between the samples according to thephase of the specific reference block derived based on L1 motion vectorand L1 reference picture index of the modified motion information andthe reconstructed block of the target block is greater than thepreconfigured threshold value, the decoding apparatus may derive themodified motion information as the uni-prediction motion informationincluding L0 motion vector and L0 reference picture index.

In addition, in the case that the differential value between the samplesaccording to the phase of the specific reference block derived based onL0 motion vector and L0 reference picture index and the reconstructedblock of the target block is greater than the preconfigured thresholdvalue and in the case that the differential value between the samplesaccording to the phase of the specific reference block derived based onL1 motion vector and L1 reference picture index and the reconstructedblock of the target block is greater than the preconfigured thresholdvalue, the decoding apparatus may derive the motion information of theneighboring block of the target block as the modified motion informationor may not calculate the modified motion information.

Meanwhile, the decoding apparatus may select the specific referencepicture indicated by the modified reference picture index included inthe modified motion information through various methods.

For example, the decoding apparatus may select the reference picturewhich is the most recently encoded among the reference pictures includedin the reference picture list L0 and generate the modified L0 referencepicture index indicating the reference picture. In addition, thedecoding apparatus may select the reference picture which is the mostrecently encoded among the reference pictures included in the referencepicture list L1 and generate the modified L1 reference picture indexindicating the reference picture.

For another example, the decoding apparatus may select the referencepicture having a POC of which an absolute value of a difference from apicture order count (POC) of the target picture among the referencepictures included in the L0 reference picture index and generate themodified L0 reference picture index indicating the reference picture. Inaddition, the decoding apparatus may select the reference picture havinga POC of which an absolute value of a difference from a picture ordercount (POC) of the target picture among the reference pictures includedin the L1 reference picture index and generate the modified L1 referencepicture index indicating the reference picture.

For another example, the decoding apparatus may select the referencepicture that belongs to the lowest layer on the hierarchical structureamong the reference pictures included in each of the reference picturelists and generate the modified reference picture index indicating thereference picture. The reference picture that belongs to the lowestlayer may be I-slice or a reference picture which is encoded by applyinga low quantization parameter (QP).

For another example, the decoding apparatus may select the referencepicture including a reference block of which reliability of the motioncompensation is the highest in the reference picture list and generatethe modified reference picture index indicating the reference picture.In other words, the decoding apparatus may derive the specific referenceblock of the reconstructed block of the target block based on thereference pictures included in the reference picture list and generatethe modified reference picture index indicating the specific referencepicture including the derived specific reference block.

The methods of generating the modified reference picture index of themodified motion information described above may be independently appliedor the methods may be applied in combination.

Meanwhile, the decoding apparatus may obtain the motion vector updatedifferential information indicating a differential between the existingmotion vector of the target block and the modified motion vector throughthe bitstream. In this case, the decoding apparatus may derive themodified motion information by summing the (first) motion information ofthe target block and the motion vector update differential information.The motion vector update differential information may be transmitted ina unit of PU described above.

The decoding apparatus updates the motion information of the targetblock based on the modified motion information (step, S1350). Thedecoding apparatus may store the modified motion information and updatethe motion information of the target block. The decoding apparatus mayupdate the motion information of the target block by substituting themotion information used in predicting the target block to the modifiedmotion information. In addition, the decoding apparatus may update themotion information of the target block by storing all the motioninformation and the modified motion information used in predicting thetarget block. The updated motion information may be used for the motioninformation of the next block neighboring the target block.

For example, the merge mode is applied to the next block neighboring thetarget block, the merge candidate list of the next block may include thetarget block. In the case that the motion information of the targetblock is stored by substituting the motion information used inpredicting the target block to the modified motion information, themerge candidate list of the next block may include the target blockindicating the modified motion information. In addition, all the motioninformation and the modified motion information used in predicting thetarget block are stored in the motion information of the target block,the merge candidate list of the next block may include the target blockindicating the motion information used in predicting the target blockand the target block indicating the modified motion information. Thetarget block indicating the modified motion information may be insertedas the spatial neighboring candidate block in the merge candidate listor inserted as the temporal neighboring candidate block in the mergecandidate list.

For another example, the AMVP mode is applied to the next blockneighboring the target block, like the method of inserting the updatedmotion information of the target block neighboring the next block in themerge candidate list in the merge mode described above as the spatial ortemporal neighboring motion information, the method of inserting theupdated motion information of the neighboring block in the motion vectorpredictor candidate list in the next block above as the spatial ortemporal motion vector predictor candidate may be applied. That is, thedecoding apparatus may generate the motion vector predictor candidatelist including the updated motion information of the target blockneighboring the next block.

For example, in the case that the updated motion information of thetarget block neighboring the next block includes only the modifiedmotion information, the motion vector predictor candidate list mayinclude the modified motion information as the spatial motion vectorpredictor candidate.

For another example, in the case that the updated motion information ofthe target block neighboring the next block includes the modified motioninformation and the existing motion information of the target block, themotion vector predictor candidate list may include the modified motioninformation selected according to a specific condition among theexisting motion information selected according to a specific conditionand the modified motion information of the target blocks neighboring thenext block among the existing motion information of the target blockneighboring the next block as the respective spatial motion vectorpredictor candidate.

Meanwhile, the motion vector predictor candidate list of the next blockmay include the updated motion information of the collocated block ofthe same position as the position of the next block and the updatedmotion information of the neighboring block of the same position blockas the temporal motion vector predictor candidate list in the referencepicture of the next block. For example, in the case that the updatedmotion information of the collocated block of the same position as theposition of the next block and the updated motion information of theneighboring block of the same position block include only the modifiedmotion information of each block, the motion vector predictor candidatelist may include the modified motion information as the temporal motionvector predictor candidate list.

For another example, in the case that the updated motion informationincludes the same position block, the respective modified motioninformation of the neighboring block of the same position block and theexisting motion information of the same position block, the motionvector predictor candidate list may include the modified motioninformation selected according to the specific condition among theexisting motion information selected according to the specific conditionand the modified motion information among the existing motioninformation as separate spatial motion vector predictor candidate,respectively.

Meanwhile, the stored motion information and the motion information fortransmission may have different resolution. For example, the decodingapparatus decodes and transmits the information for the motioninformation and the modified motion information is stored for the motioninformation neighboring the target block, the unit of motion vectorincluded in the motion information may have a unit of ¼ fractionalsample and the unit of motion vector of the modified motion informationmay represent a unit of ⅛ fractional sample or 1/16 fractional sample.

Meanwhile, although it is not shown, the decoding apparatus maydetermine whether to update the (first) motion information of the targetblock by performing the comparison procedure between the (first) motioninformation and the modified motion information based on thereconstructed block of the target block. For example, the decodingapparatus may determine whether to update the target block by comparingthe amount of data of the residual signal of the specific referenceblock derived based on the modified motion information and thereconstructed block of the target block with the amount of data of theresidual signal of the reference block derived based on the motioninformation and the reconstructed block. Among the amounts of data, inthe case that the amount of data of the residual signal of the specificreference block and the reconstructed block is smaller, the decodingapparatus may determine to update the motion information and the targetblock. In addition, among the amounts of data, in the case that theamount of data of the residual signal of the specific reference blockand the reconstructed block is not smaller, the decoding apparatus maydetermine not to update the motion information and the target block.

Meanwhile, the decoding apparatus may obtain the additional informationindicating whether to update the target block through the bitstream. Forexample, the additional information indicating whether to update may bereferred as an update flag. The case that the update block is 1 mayindicate that the motion information is updated and the case that theupdate block is 0 may indicate that the motion information is notupdated. For example, the update flag may be transmitted in a unit ofPU. Alternatively, the update flag may be transmitted in a unit of CU, aunit of CTU or a unit of slice and may be transmitted through a higherlevel such as a unit of picture parameter set (PPS) or a unit ofsequence parameter set (SPS).

According to the present invention described above, after decodingprocedure of a target block, modified motion information of the targetblock is calculated, and can be updated to more accurate motioninformation, and through this, overall coding efficiency can beimproved.

In addition, according to the present invention, motion information of anext block neighboring the target block can be derived based on theupdated motion information of the target block, and propagation ofdistortion can be decreased, and through this, overall coding efficiencycan be improved.

In the above-described embodiments, methods are described based on aflowchart as a series of steps or blocks. However, the presentdisclosure is not limited to the order of steps s. Some steps may occursimultaneously or in a different order than the order of the stepsdescribed above. Further, those skilled in the art will appreciate thatthe step shown in the flowchart is not exclusive. It will be understoodthat other steps may be included or that one or more steps in theflowchart may be deleted without affecting the scope of the presentdisclosure.

The method according to the present disclosure described above may beimplemented in software. The encoding device and/or decoding deviceaccording to the present disclosure may be included in a device thatperforms image processing, for example, for a TV, a computer, a smartphone, a set-top box, or a display device.

When the embodiments of the present disclosure are implemented insoftware, the above-described method may be implemented by modules(processes, functions, and so on) that perform the functions describedabove. Such modules may be stored in memory and executed by a processor.The memory may be internal or external to the processor, and the memorymay be coupled to the processor using various well known means. Theprocessor may comprise an application-specific integrated circuit(ASIC), other chipsets, a logic circuit and/or a data processing device.The memory may include a ROM (read-only memory), a RAM (random accessmemory), a flash memory, a memory card, a storage medium, and/or otherstorage device.

1. An image decoding method performed by a decoding apparatus, themethod comprising: deriving motion information of a first target blockin a first picture; deriving modified motion information for the firsttarget block based on the motion information and bi-prediction; when amerge mode is applied to a second target block in a second picture,configuring a merge candidate list for the second target block based onspatial neighboring blocks and a temporal neighboring block of thesecond target block; receiving a merge index for the second targetblock; selecting one of merge candidate constituting the merge candidatelist based on the merge index; deriving motion information of the secondtarget block based on the selected merge candidate; and generatingprediction sample for the second target block by performing interprediction based on the motion information of the second target block,wherein the merge candidates includes spatial motion informationcandidates and a temporal motion information candidate, wherein thetemporal motion information candidate is derived based on the temporalneighboring block, wherein when the temporal neighboring blockcorresponds to the first target block, the temporal motion informationcandidate is derived based on the modified motion information of thefirst target block.
 2. The image decoding method of claim 1, furthercomprising: determining whether to update the motion information of thefirst target block, when the motion information of the first targetblock is determined to be updated, wherein the motion information of thefirst target block is updated based on the modified motion informationof the first target block.
 3. The image decoding method of claim 1,wherein the updated motion information is used for deriving motioninformation of a next block spatially neighboring the first target blockin the first picture.
 4. The image decoding method of claim 1, whereinwhen a unit of motion vector included in the motion information of thefirst target block represents a unit of ¼ fractional sample, the unit ofmotion vector included in the modified motion information of the firsttarget block represents a unit of ⅛ fractional sample
 5. The imagedecoding method of claim 1, wherein the step of deriving the modifiedmotion information includes: deriving a specific reference picture forthe modified motion information among the reference pictures; detectinga specific reference block among the reference blocks in the specificreference picture; and deriving the modified motion informationincluding the modified reference picture index indicating the specificreference picture and the modified motion vector indicating the specificreference block.
 6. The image decoding method of claim 5, wherein thespecific reference block is derived as a reference block of which sum ofdifference (SAD) from the reference block is a minimum among thereference blocks in the specific reference picture, wherein the SAD isdetermined based on the following equation:${SAD} = {\sum\limits_{j}^{height}{\sum\limits_{i}^{width}{{{{Block}_{cur}\left( {i,j} \right)} - {{Block}_{ref}\left( {i,j} \right)}}}}}$herein, Block_(cur)(i,j) represents a reconstructed sample of (i, j)coordinate in the reconstructed block, Block_(ref)(i,j) represents areconstructed sample of (i, j) coordinate in the reference block,‘width’ represents a width of the reconstructed block and ‘height’represents a height of the reconstructed block.
 7. The image decodingmethod of claim 5, wherein the specific reference picture for themodified motion information is derived as a reference picture indicatedby the reference picture index included in the motion information, andwherein the reference block of which SAD of the reference block isderived as the specific reference block among the reference blocksincluded in a predetermined area from the reference block which isderived from the specific reference picture based on the motion vectorincluded in the motion information.
 8. The image decoding method ofclaim 1, further comprising: when a size of a first target block isgreater than a preconfigured size, partitioning the first target blockinto a plurality of sub blocks; and deriving a sub reference block in aunit of each sub block in a reference picture, wherein modified motioninformation for the sub blocks in the first target block is derivedbased on the derived sub reference blocks.
 9. The image decoding methodof claim 1, wherein the modified motion information for the first targetblock is derived through an optical flow (OF) method performed based onL0 motion vector and L1 motion vector included in the motioninformation.
 10. The image decoding method of claim 1, wherein themodified motion information is bi-prediction motion information, andwherein the bi-prediction motion information includes L0 motion vectorand L0 reference picture index, and L1 motion vector and L1 referencepicture index.
 11. (canceled)
 12. The image decoding method of claim 10,wherein the L0 reference picture index of the modified motioninformation indicates a reference picture having a picture order count(POC) of which absolute value of a difference from POC of the firstpicture in which the first target block is included is a smallest amongthe reference pictures included in the reference picture list L0, andwherein the L1 reference picture index of the modified motioninformation indicates a reference picture having a picture order count(POC) of which absolute value of a difference from POC of the firstpicture in which the first target block is included is a smallest amongthe reference pictures included in the reference picture list L1. 13.(canceled)
 14. (canceled)
 15. (canceled)
 16. An image decodingapparatus, the apparatus comprising: a predictor configured to derivemotion information of a first target block in a first picture, to derivemodified motion information for the first target block based on themotion information and bi-prediction, to configuring a merge candidatelist for the second target block based on spatial neighboring blocks anda temporal neighboring block of the second target block when a mergemode is applied to a second target block in a second picture; and anentropy decoder configured to receive a merge index for the secondtarget block; wherein the predictor configured to select one of mergecandidate constituting the merge candidate list based on the mergeindex, to derive motion information of the second target block based onthe selected merge candidate, and to generate prediction sample for thesecond target block by performing inter prediction based on the motioninformation of the second target block, wherein the merge candidatesincludes spatial motion information candidates and a temporal motioninformation candidate, wherein the temporal motion information candidateis derived based on the temporal neighboring block, wherein when thetemporal neighboring block corresponds to the first target block, thetemporal motion information candidate is derived based on the modifiedmotion information of the first target block.
 17. An image encodingmethod performed by an encoding apparatus, the method comprising:deriving motion information of a first target block in a first picture;deriving modified motion information for the first target block based onthe motion information and bi-prediction; when a merge mode is appliedto a second target block in a second picture, configuring a mergecandidate list for the second target block based on spatial neighboringblocks and a temporal neighboring block of the second target block;selecting one of merge candidate constituting the merge candidate listbased on the merge index; generating merge index for the second targetblock indicating the selected merge candidate; and encoding imageinformation including the merge index, wherein the merge candidatesincludes spatial motion information candidates and a temporal motioninformation candidate, wherein the temporal motion information candidateis derived based on the temporal neighboring block, wherein when thetemporal neighboring block corresponds to the first target block, thetemporal motion information candidate is derived based on the modifiedmotion information of the first target block.